Papers with fully attention-based approach
A Multiscale Visualization of Attention in the Transformer Model (P19-3)
Copied to clipboard
| Challenge: | Various tools have been developed to visualize attention in NLP models, ranging from attention-matrix heatmaps to bipartite graph representations. |
| Approach: | They propose an open-source tool that visualizes attention at multiple scales and provides a unique perspective on the attention mechanism. |
| Outcome: | The proposed model outperforms OpenAI GPT-2 and BERT on several language modeling benchmarks. |